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1 . This action is responsive to Amendment A filed on March/31/2004. 

2. Claims 17-51 are presented for examination. Claims 1-16 have been canceled. 

Claim Rejections - 35 USC § 102 

(e) the invention was described in a patent granted on an application for patent by another filed in the 
United States before the invention thereof by the applicant for patent, or on an international application 
by another who has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371 (c) of this 
title before the invention thereof by the applicant for patent. 

The changes made to 35 U.S.C. 102(e) by the American Inventors Protection Act 
of 1999 (AIPA) and the Intellectual Property and High Technology Technical 
Amendments Act of 2002 do not apply when the reference is a U.S. patent resulting 
directly or indirectly from an international application filed before November 29, 2000. 
Therefore, the prior art date of the reference is determined under 35 U.S.C. 102(e) prior 
to the amendment by the AIPA (pre-AlPA 35 U.S.C. 102(e)). 

3. Claim 17-31 , 38, 44-45, 47 an£&f are rejected under 35 U.S.C. 102 (e) as being 
anticipated by applicant submitted IDS issued to 1999 IEEE Shadow Gestures: 3D 
Hand Pose Estimation using a Single Camera. 

As per claims 17 (method) and 24 (device), Shadow Gestures discloses a 
method for recognizing a selection a set of at least two alternatives as the technique of 
moving the thumb and the index finger (see page 484, right col., lines 1-2), the method 
comprising: 



Application/Control Number: 09/879,438 Page 3 

Art Unit: 2173 

Determining the positions corresponding to each alternative in the space 
surround a user on the basis of their distance and direction with respect to the user so 
that the locations of the positions remain substantially the same with respect to user 
irrespective of the location of the user is taught by Shadow Gestures as the technique of 
a light source to cast the shadow of a hand and use a single camera to image both the 
hand and its shadow. Features derived from projections of the hand and the shadow are 
then used to compute 3D position and orientation (see page 479, right col., lines 17-21) 
and the geometry of the shadow formation calculation (see page 483, left col., line 23 to 
right col., line 39 and see Fig. 7); 

Allowing the user to carry out a first movement for moving a member of the body 
to a position corresponding to an alternative the user desires is taught by Shadow 
Gestures as the technique of user controls the robot arm by moving the thumb (see 
page 484, right col., line 1); 

Recognizing a second movement carried out by the user in the position 
corresponding to the alternative the user desires is taught by Shadow Gestures as the 
technique of user controls the robot arm by moving the thumb and the index finger (see 
page 484, right col., lines 1-2); 

In response to the second movement, recognizing the selection the user desires 
as completed is taught by Shadow Gestures as the technique of selecting simple 
primates from a menu and manipulating them (see page 484, left col., lines 50-51 and 
see Fig. 8); 
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Providing the recognizing selection as an output is taught by Shadow Gestures 
as 3D Scene in Fig. 8. 

These claims are therefore rejected for the reasons as set forth above. 

As per claim 45 (system), due to the mostly similarity of this claim to that of claim 
17 (method), except for the system comprising: a central processing unit, a three 
dimensional display device, the central unit comprising communication means for 
communicating positions corresponding to selection alternatives to the three 
dimensional display device are taught by Shadow Gestures as the techniques of our 
system operates at 60 HZ on a SGI Indy (see page 479, right col., line 40) to compute 
3D position and orientation (see page 479, right col., line 21) for the user control up to 
seven parameters (gripper's position, orientation and jaw-separation) by natural finger 
movement (see page 479, left col., lines 20-22). This claim is therefore rejected for the 
reasons as set forth above. 

As per claims 18 (method) and 25 (device), the limitation of indicating the user at 
least once the positions corresponding to the alternatives as showing virtual images in 
each position is taught by Shadow Gestures as the technique of virtual fly-thru's over 
terrains (see page 484, right col., lines 7-8 and see Fig. 9). These claims are therefore 
rejected for the reasons as set forth above. 
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As per claims 19 (method) and 26 (device), the limitation of demonstrating the 
user the alternative indicated at any given time is taught by Shadow Gestures as the 
technique of real-time gesture recognition and hand tracking system that can be used 
as input interface to applications that require multi dimensional control (see page 479, 
left col., lines 26-29). These claims are therefore rejected for the reasons as set forth 
above. 

As per claims 20 (method) and 27 (device), the limitation of recognizing the 
second movement contactlessly is taught by Shadow Gestures as the technique of 
virtual fly-thru's over terrains (see page 484, right col., lines 7-8 and see Fig. 9). These 
claims are therefore rejected for the reasons as set forth above. 

As per claims 21 (method) and 28 (device), the limitation of wherein the first 
movement is the movement of the user's hand is taught by Shadow Gestures as the 
technique of user controls the robot arm by moving the thumb and the index finger (see 
page 484, right col., lines 1-2). These claims are therefore rejected for the reasons as 
set forth above. 

As per claim 22 (method), the limitation of carry out the first function to the output 
is taught by Shadow Gestures as the technique of controlling robot arm by moving the 
thumb (see page 484, right col., lines 1-2 and see 3D Scene in Fig. 8). This claim is 
therefore rejected for the reasons as set forth above. 
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As per claim 29 (device), the limitation of carrying out a first function in response 
to the second movement is taught by Shadow Gestures as the technique of controlling 
robot arm by moving the thumb and index finger (see page 484, right col., lines 1-2 and 
see 3D Scene in Fig. 8). 

This claim is therefore rejected for the reasons as set forth above. 

As per claims 23 (method) and 30 (device), the limitation of allowing the user to 
carry out certain second activity with a specific third movement of the member of the 
body is taught by Shadow Gestures as the technique of an interface to 3D video games 
where users can navigate by finger-pointing. The user can also fire guns using the 
"click" gesture and open doors with the " reach" gesture (see page 484, right col., line 
12 to page 485, left col., line 2). These claims are therefore rejected for the reason as 
set forth above. 

As per claim 31, the limitation of recognizing the second movement carried out 
by the user in the position are adapted to be attached to the user is taught by Shadow 
Gestures as the technique of user controls the robot arm by moving the thumb and the 
index finger (see page 484, right col., lines 1-2) for selecting simple primates from a 
menu and manipulating them (see page 484, left col., lines 50-51) of 3D video games 
(see page 484, right col., line 13). This claim is therefore rejected for the reason as set 
forth above. 
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As per claims 38 (method) and 44 (device), the limitation of determining the 
positions corresponding to each alternative in the space surrounding a user also on the 
basis of their distance with respect to the user is taught by Shadow Gestures as the 
technique of a light source to cast the shadow of a hand and use a single camera to 
image both the hand and its shadow. Features derived from projections of the hand and 
the shadow are then used to compute 3D position and orientation (see page 479, right 
col., lines 17-21) and the geometry of the shadow formation calculation (see page 483, 
left col., line 23 to right col., line 39 and see Fig. 7). These claims are therefore rejected 
for the reasons as set forth above. 

As per claim 47, the limitation of recognizing is a camera is taught by Shadow 
Gestures as the technique of we presented a one camera system that recognizes three 
gestures and tracks the user's hand (see page 485, left col., lines 31-32). This claim is 
therefore rejected for the reasons as set forth above. 

As per claim 50, the limitation of wherein the three dimensional display device 
and the means for recognizing are comprised in the same unit is taught by Shadow 
Gestures as the technique of we presented a one camera system that recognizes three 
gestures and tracks the user's hand in 3D (see page 485, left col., lines 31-32). This 
claim is therefore rejected for the reasons as set forth above. 
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Claim Rejections - 35 USC § 103 



4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 32, 46, and 48 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over in view of applicant submitted IDS issued to 1999 IEEE Shadow 
Gestures: 3D Hand Pose Estimation using a Single Camera in view of Kumar et al. 
(USPN: 6,624,833) hereinafter Kumar. 

As per claims 32 (device) and 46 (system), Shadow Gestures discloses the 
invention substantially as claimed above. Shadow Gestures, however, does not disclose 
the limitation of wherein the device comprises at least one of the following: mobile 
station, a computer, a television apparatus, a data network browsing device, an 
electronic book, and an at least partly electronically controlled vehicle. 

Kumar discloses the limitation of wherein the device comprises a computer as 
the technique of computer system 12 (see col. 5, line 32). 

It would have obvious to one having ordinary skill in the art at the time the 
invention was made to include Kumar's teaching of computer system into that of 
Shadow Gestures' invention. By doing so, the system would be enhanced by capable of 
connecting to a device and allowing user to select alternative choice and perform some 
functional tasks on virtual environment. 
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As per claim 48, Shadow Gestures discloses the invention substantially as 
claimed above. Shadow Gestures, however, does not disclose the limitation of wherein 
the means for recognizing is a shape tape. 

Kumar discloses the limitation of recognizing is a shape tape as the technique of 
the system 10 can of course be utilizing with other types of information processing 
devices (see col. 5, lines 33-34). 

It would have obvious to one having ordinary skill in the art at the time the 
invention was made to include Kumar's teaching of recognizing is a shape tape into that 
of shadow Gestures' invention. By doing so, the system would be enhanced by capable 
of allowing user to select shape tape device for recognizing and recording user 
movement in virtual environment. 

6. Claims 33-37, 39-43 and 49 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over in view of applicant submitted IDS issued to 1999 IEEE Shadow 
Gestures: 3D Hand Pose Estimation using a Single Camera in view of Selker (USPN: 
6,549,219). 

As per claims 33 (method) and 39 (device), Shadow-Gestures discloses the 
invention substantially as claimed above. Shadow-Gestures, however, does not 
disclose the limitation of wherein the positions are sectors on an arcuate area. 

Selker discloses the positions are sectors on an arcuate as the technique of a 
pie menu wherein highest popularity based on a historical analysis is placed in the level 
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1 circle 10. Level 2 receives a secondary menu item 20 of less importance (see col. 3, 
lines 39-41 and also see Fig. 1). 

It would have obvious to one having ordinary skill in the art at the time the 
invention was made to include Selker's teaching of the positions are sectors on an 
arcuate area into that of shadow Gestures' invention. By doing so, the system would be 
enhanced by capable of allowing user to select any desired menu item in the pie menu 
based on user desired. 

As per claims 34 (method), 40 (device), and 49 (system); Shadow-Gestures 
discloses the invention substantially as claimed above. Shadow-Gestures, however, 
does not disclose the limitation of wherein arcuate area is selection disc. 

Selker discloses the limitation of wherein arcuate area is selection disc as the 
technique of a multiple level pie menu. As with the concentric menu system of Fig. 1, 
the menus are separate into multiple levels of importance. However, in this concentric 
level 1 is split into multiple menu selections 11, 12, 13, and 14 (see col. 3, lines 44-48 
and see Fig. 2). 

It would have obvious to one having ordinary skill in the art at the time the 
invention was made to include Selker's teaching of wherein arcuate area is selection 
menu disc into that of Shadow Gestures' invention. By doing so, the system would be 
enhanced by capable of allowing user option to select any desired menu item in the pie 
menu based on user desired. 



Application/Control Number: 09/879,438 Page 1 1 

Art Unit: 2173 

As per claims 35 (method) and 41 (device), Shadow Gestures discloses the 
invention substantially as claimed above. While Shadow Gestures discloses the first 
movement is a substantially horizontal movement of the hand as the technique of the 
user stretches out the thumb (see page 484, right col., lines 8-9). Shadow Gestures, 
however, does not disclose the limitation of the first movement is a substantially 
horizontal movement of the hand to a certain sector of an arcuate area situated 
substantially in horizontal plane . 

Selker discloses a certain sector of an arcuate area situated substantially in 
horizontal plane as the technique of a multiple level pie menu. As with the concentric 
menu system of Fig. 1, the menus are separate into multiple levels of importance. 
However, in this concentric level 1 is split into multiple menu selections 1 1 , 12, 13, and 
14 (see col. 3, lines 44-48 and see Fig. 2). 

It would have obvious to one having ordinary skill in the art at the time the 
invention was made to combine Shadow Gestures 1 teaching of the first movement is a 
substantially horizontal movement of the hand into that of Selker's certain sector of pie 
menu arcuate area invention. By doing so, the system would be enhanced by capable 
of allowing user option to perform stretch out movement of the thumb and select any 
desired menu item in the pie menu based on user desired manner. 



As per claims 36 (method) and 42 (device), Shadow Gestures discloses the 
invention substantially as claimed above. While Shadow Gestures discloses the second 
movement is a substantially vertical movement of the hand as the technique of the user 
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stretches out the thumb and the pointing finger and initiates flying by moving the hand 
(see page 484, right col., lines 8-10). Shadow Gestures, however, does not disclose the 
limitation of the second movement is a substantially vertical movement of the hand at 
said certain sector. 

Selker discloses the limitation of certain sector as the technique of a pie menu 
wherein highest popularity based on a historical analysis is placed in the level 1 circle 
10. Level 2 receives a secondary menu item 20 of less importance (see col. 3, lines 39- 
41 and also see Fig. 1). 

It would have obvious to one having ordinary skill in the art at the time the 
invention was made to combine Shadow Gestures' teaching of the second movement is 
a substantially vertical movement of the hand into that of Selker's certain sector of pie 
menu arcuate area invention. By doing so, the system would be enhanced by capable 
of allowing user option to perform fly through movement of the hand and select any 
desired menu item in the pie menu based on user desired manner. 

As per claims 37 (method) and 43 (device), Shadow Gestures discloses the 
invention substantially as claimed above. While Shadow Gestures discloses the second 
movement as the technique of the user stretches out the thumb and the pointing finger 
and initiates flying by moving the hand (see page 484, right col., lines 8-10). Shadow 
Gestures, however, does not disclose the limitation of the second movement is placing 
a hand into a certain position at said certain sector. 



Application/Control Number: 09/879,438 Page 13 

Art Unit: 2173 

Selker discloses the limitation of placing a hand into a certain position at said 
certain sector as the technique of each menu item is selected by direct placement of a 
cursor over the item and activation by an input device input such as a right mouse click 
or double click (see col. 5, lines 6-8). 

It would have obvious to one having ordinary skill in the art at the time the 
invention was made to combine Shadow Gestures' teaching of the second movement 
into that of Selker's certain sector of pie menu arcuate area invention. By doing so, the 
system would be enhanced by capable of allowing user option to perform second 
movement and select any desired menu item in the pie menu based on user desired 
manner. 

7. Claim 51 is rejected under 35 U.S.C. 103(a) as being unpatentable over in view 
of applicant submitted IDS issued to 1999 IEEE Shadow Gestures: 3D Hand Pose 
Estimation using a Single Camera in view of Kumar et al. (USPN: 6,624,833) hereinafter 
Kumar and further in view of applicant submitted IDS issued to Tomi Engdahl,1999: 
3Dglasses and other 3D display device. 

As per claim 51 , Shadow Gestures-Kumar disclose the invention substantially as 
claimed above. Shadow Gestures-Kumar, however, do not disclose the limitation of the 
three dimensional display device is virtual glass. 

Engdahl disclose the limitation of the three dimensional display device is virtual 
glass as the technique of the 3D material can be stored to any standard color video 
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media and viewed with normal display devices as long as you wear the right color filter 
glasses (see page 1 , lines 30-31). 

It would have obvious to one having ordinary skill in the art at the time the 
invention was made to include Engdahrs teaching of virtual 3D glass into that of 
Shadow Gestures-Kumar combined invention. By doing so, the system would be 
enhanced by capable of allowing user to use 3D virtual glasses for viewing color video 
media. 

8. Applicants' arguments filed on March/31/2004 have been fully considered, but 
they are not persuasive. 

On page 13, first paragraph, Applicants argues that " Shadow Gestures relates to 
multi-dimensional control and does not even mention different positions corresponding 
to different alternatives. Instead, in Shadow Gestures different fingers gestures 
correspond to different control commands ". The Examiner, however, does not agree to 
this argument since in Shadow Gestures 1 invention, different positions is taught as the 
technique of Point, Reach, Click and Ground (see page 480, left col.) corresponds up to 
seven parameters of gripper's position, orientation and jaw-separation (see page 479, 
left col.), this seven parameters of gripped position, orientation and jaw-separation 
correspond to different alternative based on user's movement activities. 

On page 13, second paragraph, Applicants ague that " Shadow Gestures does 
not teach recognizing user's selection on the basis of a second movement or that a 
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second movement is done in the position corresponding to the desired alternative. 
Instead, Shadow Gestures recognizes control commands on the basis of finger 
gestures, and no sequence of movements, wherein the second movement is done in 
the position corresponding to the desired alternative.". The Examiner, however, does 
not agree to this argument since in Shadow Gestures' invention, the second movement 
and the sequence of movement are taught as the technique of user controls the robot 
arm by moving the thumb and the index finger (see page 484, right col., lines 1-2), the 
user stretches out the thumb and the pointing finger and initiates flying by moving the 
hand (see page 484, right col., lines 8-10). Thus by moving the index finger and initiate 
consequence of flying the hand, the user can control of hand and arm movement in 
virtual three-dimensional environment. 

On page 13, third paragraph, Applicants ague that "the method of claim 17 allows 
the use of rough hand and arm movements in making a selection". The Examiner, 
however, does not agree to this argument since the feature of "allows the use of rough 
hand and arm movements in making a selection" is taught by Shadow Gestures as the 
technique of user controls the robot arm bv moving the thumb and the index finger (see 
page 484, right col., lines 1-2), the user stretches out the thumb and the pointing finger 
and initiates flying bv moving the hand (see page 484, right col., lines 8-10). 

On page 13, fourth paragraph, Applicants ague that " Claims 24 and 45 have 
similar limitations in structure form." The Examiner, however, does not agree to this 
argument. These claims are rejected for the same reasons applied to claim 17. 
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On page 13, fifth paragraph, Applicants ague that " Thus claims 17-31, 33-45, 
and 47-51 define over Shadow Gestures. Further since the above features are not 
suggested by this reference, these claims are unobvious over it". The Examiner, 
however, does not agree to this argument. Claims 17-31, 33-45, and 47-51 are rejected 
for at least of the same reasons based on Shadow Gestures as set forth above. 

On the last paragraph of page 13, Applicants ague that " Kumar fails to disclose 
the above features. Thus combining it with Shadow Gestures does not result in the 
claimed invention. Thus claims 32 and 46 are unobvious under 35 USC 103 on this 
reference combination.". The Examiner, however, does not agree to this argument since 
as indicated by the Examiner: Shadow Gestures does not disclose the limitation of 
wherein the device comprises at least one of the following: mobile station, a computer, a 
television apparatus, a data network browsing device, an electronic book, and an at 
least partly electronically controlled vehicle. 

Kumar discloses the limitation of wherein the device comprises a computer as 
the technique of computer system 12 (see col. 5, line 32). 

It would have obvious to one having ordinary skill in the art at the time the 
invention was made to include Kumar's teaching of computer system into that of 
Shadow Gestures 1 invention. By doing so, the system would be enhanced by capable of 
connecting and communicating to a device and allowing user to select alternative 
choice and perform some functional tasks on virtual environment. 
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Conclusion 



9. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CUONG T THAI whose telephone number is (703) 308- 
7234. The examiner can normally be reached on 8:00 am - 4:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John W. Cabeca can be reached on (703) 308-31 16. The fax phone 
number for the organization where this application or proceeding is assigned is 703- 
872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



CUONG T THAI 
Examiner 
Art Unit 2173 



July/08/2004. 
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